There are cameras known in the related art, that have a blur correcting function for preventing an unsteady hand movement during a photographing operation from lowering the quality of the captured image. Blurring is corrected in such a camera by adopting one of the following two primary methods.
The first method is an optical blur correction method in which a vibration of the camera is detected by using a vibration detection sensor such as an angular velocity sensor or an acceleration sensor and a blur is corrected by driving an optical system such as a photographic lens or a variable apex-angle prism in correspondence to the extent of the detected vibration (see, for instance, Japanese Laid Open Patent Publication No. S61-240780).
The second method is an electronic blur correction method in which the extent of blur is determined based upon the difference between the captured image and a previous image having been stored in memory on a temporary basis and the blur is corrected when reading out the image (see, for instance, Japanese Laid Open Patent Publication No. S63-187883). Through either of these two methods, the blur is corrected in real-time when the image is photographed.
There is another technology known in the related art which is used as an alternative blur correction method to those described above, through which a degraded image is restored as a blur-free image, unaffected by any unsteady hand movement. For instance, Japanese Laid Open Patent Publication No. S62-127976 discloses a method in which degradation of an image caused by a vibration occurring during the photographing operation is expressed as a point spread function and the image is restored as a blur-free image based upon the point spread function. There is also a technology known in the related art adopted in conjunction with a camera equipped with a vibration detection means alone, through which hand movement information is recorded and a blur is corrected by executing image restoration processing based upon the information when reproducing the image (see, for instance, Japanese Laid Open Patent Publication No. H 6-276512).
A specific method adopted in the image restoration processing is now explained. The term “image restoration” refers to a restoration of a blurred image, achieved by processing the blurred image based upon blur-related information so as to obtain an image manifesting a lesser extent of blurring.
With (x, y) representing positional coordinates on an image plane, o(x, y) representing an image obtained without experiencing any vibration (hereafter referred to as a raw image), z(x, y) representing an image degraded due to vibration (hereafter referred to as a blurred image) and p(x, y) representing information of a point image having become spread due to vibration (hereafter referred to as a point spread function), o(x, y), z(x, y) and p(x, y) achieve a relationship expressed as follows;z(x,y)=o(x,y)*p(x, y,)  (1)In the expression above, “*” indicates a convolution (convoluted integration) arithmetic operation, which is expressed specifically as follows;z(x, y)=∫∫σ(x, y)p(x−x′, y−y′)dx′ dy′  (2)When the relationship is transformed into a relationship in a spatial frequency (u, v) range through a Fourier transform, expressions (1) and (2) are rewritten as follows;Z(u, v)=O(u, v)·P(u, v)  (3)
Z(u, v), O(u, v) and P(u, v) respectively represent the spectrums of z(x, y), o(x, y) and p(x, y). In addition, P(u, v) in expression (3) is specifically referred to as a spatial frequency transfer function.
If the point-image function p(x, y) can be somehow ascertained in addition to the blurred image z(x, y), their spectrums can be computed and then the spectrum O(u, v) of the raw image can be computed by using the following expression (4), which is a modified form of expression (3).
                              O          ⁡                      (                          u              ,              v                        )                          =                              Z            ⁡                          (                              u                ,                v                            )                                            P            ⁡                          (                              u                ,                v                            )                                                          (        4        )            
1/P(u, v) in expression (4) is specifically referred to as an inverse filter. The raw image o(x, y) can be determined through an inverse Fourier transformation of the spectrum computed by using expression (4). FIGS. 6(a) to 6(c) and FIGS. 7(a) to 7(d) illustrate the image restoration executed in the related art.
In order to simplify the explanation, it is assumed that a uniform blur has occurred along a single axis (the X axis), as shown in FIG. 6(b).
FIG. 7 (a) shows a section taken from the point spread function. The results of a Fourier transformation executed on this section in FIG. 7(a), which are shown in FIG. 7(b), constitute the spatial frequency transfer function of the blur shown in FIG. 6(a). This transfer function has characteristics of special interest in that it assumes the value 0 at a plurality of points. The inverse filter of this function manifests instances of infinity, as shown in FIG. 7(c). When the inverse filter is incorporated in expression (4), the phenomenon expressed as in (5) below occurs with regard to a specific spatial frequency and, in such a case, the spectrum value of the raw image is indeterminate.
                                          O            ⁡                          (                              u                ,                v                            )                                =                                                    Z                ⁡                                  (                                      u                    ,                    v                                    )                                                            P                ⁡                                  (                                      u                    ,                    v                                    )                                                      =                                          0                0                            =              indeterminate                                      ⁢                                                      (        5        )            
When the transfer function indicates the value 0, there is a frequency component that has not been transferred in the case of a blur (information has been lost), and accordingly, the expression above indicates that the lost frequency component cannot be restored. This, in turn, means that the complete recovery of the raw image is not possible.
It is to be noted that a Wiener filter expressed as below is actually used in the image restoration so as to ensure that the inverse filter does not manifest infinity.
                                                        P              *                        ⁡                          (                              u                ,                v                            )                                                                                                            P                  ⁡                                      (                                          u                      ,                      v                                        )                                                                              2                        +                          1              /              c                                      ⁢                                  ⁢        C        ⁢                  :                ⁢                                  ⁢        constant                            (        6        )            
FIG. 7(d) is a graph of the Wiener filter.
The use of the Wiener filter ensures that O(u, v) is not allowed to become indeterminate, unlike in expression (5).
However, the following problems exist in the optical blur correction and the image restoration in the related art described above.
(Problems of Optical Blur Correction)
An angular velocity sensor is normally used to detect vibration in the optical blur correction. In order to convert the angular velocity detected with the angular velocity sensor to an angle, the value (reference value) output from the sensor while it is in a resting state during the operation is needed. However, this reference value is known to be readily affected by drift attributable to temperature changes. This issue is now explained in detail in reference to FIGS. 8(a) and 8(b).
FIGS. 8(a) and 8(b) show the angular velocity sensor output containing the drift component, reference value outputs and the extent of blur manifesting on the image surface.
FIG. 8(a) shows the change occurring in the angular velocity sensor output value over time and in order to simplify the explanation, it is assumed that a vibration due to a hand movement, represented as a sine wave, has occurred. In FIG. 8(a), a waveform e0 indicates the vibration sensor output when a vibration due to a hand movement represented as a sine wave has occurred. In addition, waveforms e1 and e2 each represent a reference value computed through a low pass filter, with the cutoff frequency in the waveform e1 set lower than in the waveform e2. The output value in FIG. 8(a) indicates that the center of the vibration shifts (drifts) as time elapses due to environment-related factors.
FIG. 8(b) shows the extents of blurs in the image surface manifesting after executing blur correction based upon the angular velocity sensor output and the reference values in FIG. 8(a). Waveforms f0, f1 and f2 in FIG. 8(b) respectively correspond to the waveforms e0, e1 and e2 in FIG. 8(a), with the waveform f0 representing the extent of blur manifesting in the image surface when no blur correction has been executed. The waveform f1 indicates that by using the reference value e1 with a lower cutoff frequency than that in the waveform f2, the high frequency component is clipped more effectively but the extent of blur increases over time. The waveform f2, on the other hand, indicates that while the drift is reduced compared to that manifesting in the waveform f1 by using the reference value with a higher cutoff frequency, the high-frequency component attributable to the hand movement cannot be eliminated.
As described above, the requirements that need to be satisfied to eliminate an image blur caused by an unsteady hand movement and the requirements that need to be satisfied to reduce the extent of the adverse effect of drift conflict with each other, and it is difficult to select an optimal cutoff frequency for the low pass filter, at which the image blur can be corrected to a desired extent and the effect of the drift, too, is minimized. For this reason, a detection error is bound to manifest in the detected vibration extent, which gives rise to a problem in that blurring is not completely eliminated from the image having undergone the optical blur correction.
In addition, an optical blur correction apparatus often includes a switch operated to switch on/off a blur correction operation, and if the user fails to turn on the switch and a blur correction is not executed during the photographing operation, a blurred image will result.
(Problems of Image Restoration)
Next, the problems of image restoration are explained.
It is known in the related art that the resolution of an image obtained through restoration processing executed on a blurred image by using a Wiener filter is improved over that of the raw image. However, since the filter value is fairly large at a spatial frequency (u′, v′) at which P(u′, v′)≈0, the noise component is amplified if the noise contained in the image includes the spatial frequency component. This gives rise to a problem in that the image quality is lowered by an unnecessary stripe pattern that is bound to manifest in the image. While this stripe pattern does not pose a very serious problem as long as the initial blurring is insignificant, it manifests prominently if the extent of blurring is significant and in such a case, the stripe pattern becomes problematic.
In addition, cameras having an image restoration processing function in the related art are not capable of optically correcting blur but simply record output data from a vibration sensor such as an angular velocity sensor and execute restoration processing based upon the vibration information when reproducing the image. Thus, there is a problem in that if an image blur occurs to a great extent, the image quality cannot be improved through the image restoration processing due to the adverse effect of the stripe pattern described above and the like.
Furthermore, while the point-image function needed in the image restoration processing is computed based upon information such as the angular velocity sensor output and an image restoration computation is executed based upon the results of the computation of the point-image function, the volume of data output from the angular velocity sensor is extremely large, which necessitates lengthy arithmetic operations to be executed to result in poor computation efficiency. There is another problem in that it requires a high-speed arithmetic processing unit.
Moreover, even when the data needed for the image restoration are recorded into a recording medium or are transmitted to an external recipient without executing the point-image function computation or the image restoration computation, the great volume of data requires a large-capacity recording medium and a high-speed recording means or a high-speed communication means. Thus, the image restoration cannot be realized with ease and the implementation of the image restoration may lead to an increase in the cost.